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\ ABSTRACT 

VO)  ?(« i 

An  important  component  of  a large  scale  linear  programming 
system  is  the  reinversion {routine.  This  paper  addresses  an  im- 
portant ancillary  technique  for  implementing  a reinversion  routine 


utilizing  the  well-known^  and  ;jV  pivot  agenda  algorithms  of  Hellerman 


and  Rarlck.  Production  of  factors  during  reinversion  typically 
involves  a left-to-right  pivoting  process.  Unfortunately,  during 
the  left-to-right  process,  a proposed  pivot  element  of  a spike  column 
may  be  zero,  in  which  case  columns  are  interchanged  in  an  attempt  to 
obtain  a pivotable  column.  In  this  paper  we  prove  that  the  only  columns 
which  need  be  considered  for  the  interchange  with  a nonpivotable  spike 
are  ocher  spikes  lying  Co  Che  right  within  che  same  external  bump. 
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I.  INTRODUCTION 


Production  linear  programming  codes  maintain  the  Inverse  of 
the  basis  In  a factored  form  as  the  product  of  elementary  column 
matrices  (l.e.  matrices  which  differ  from  an  identity  In  one  column) 
or  elementary  row  matrices  (l.e.  matrices  which  differ  from  an  identity 
In  one  row)  or  some  combination  of  the  two.  The  vector  (either  the 
column  or  the  row)  which  distinguishes  these  matrices  from  an  identity 
matrix  are  known  as  zXa.  vcatoAA  and  the  sequence  of  these  vectors  is 
known  as  the  ETA  file.  Each  basis  change  results  in  appending  at 
least  one  eta  vector  to  the  ETA  file.  Since  both  the  time  per  pivot 
and  numerical  error  increase  as  the  length  of  the  ETA  file  increases, 
it  becomes  necessary  to  periodically  obtain  a new  factorization  of 
the  basis  inverse.  For  an  m column  basis,  this  is  accomplished  in 
m steps  by  beginning  with  an  identity  and  successively  pivoting  into 
the  identity,  columns  of  the  basis.  At  the  termination  of  the  process 
the  ETA  file  will  represent  the  inverse  of  the  current  basis.  If  in 
this  process  a proposed  pivot  element  is  zero,  then  columns  of  the 
basis  must  be  interchanged  until  a nonzero  pivot  element  can  be  found. 
Although  reinversion  techniques  have  been  discussed  in  the  literature 
(e.g.  [3,  4,  5])  a strategy  for  swapping  columns  has  never  been  explicit- 
ly detailed.  The  objective  of  this  paper  is  to  present  new  results 
which  address  the  problem  of  column  swapping.  We  prove  that  the  choices 
for  the  column  interchanges  can  be  limited  to  a few  special  columns. 
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II.  PRODUCT  FORM  OF  THE  INVERSE 


Let  B be  any  m x m nonsingular  matrix.  One  of  the  most  common 
factorizations  for  B ^ Is  the  product  form  which  corresponds  to  the 
method  for  solving  a system  of  linear  equations  known  as  Gauss-Jordan 
reduction  (see  Section  5.2  of  [1]).  This  procedure  is  used  to  represent 
B ^ as  the  product  of  matrices  each  of  the  form 


. j-ii 


:j+i; 


A few  observations  concerning  2 are  obvious. 


Proposition  1. 


Z is  nonsingular  if  and  only  if  j 0. 
Proposition  2. 


Let  0 be  any  m-component  vector  having  0^  - 0.  Then  Z0  * 0. 


Proposition  3. 


Let  0 be  any  m-component  vector  having  0^  i4  0,  and  let  e^  denote 
the  vector  having  j*"*1  component  1 and  all  other  components  zero. 


Let  zk  - 


-0k/0r  if  k * j 
l/0j  > if  k - j 


Then  Z0  - eJ 


We  now  give  the  product  form  algorithm  for  B . For  this  presen- 
tation we  adopt  the  notation  that  A(i)  denotes  the  iC^  column  of  the 


matrix  A. 


ALG  U PRODUCT  FORM  FACTORIZATION 


0.  Inlteallzatlon 

Interchange  column  of  B,  if  necessary,  so  that  the  first  component 
of  B(l)  is  nonzero.  Set  i 1,  6 B(l),  and  go  to  3. 

1.  Update.  Column 

Set  B ♦ Ei_1  •••  E1B(i). 

2.  Sump  Column a Itf  Pivot  Element  Equate  Zeno 

If  i 0,  go  to  3;  otherwise,  there  is  some  column  B(j)  with 

j > 1 such  that  the  iC^  component  of  E*  ^ •••  E^B(j)  is  nonzero. 
Interchange  B(j)  and  B(i)  and  return  to  1. 

3.  Obtain  New  Column  Eta 

Set  E*(k)  «-  ek,  for  all  k i*  i,  and 


1/B^  for  k - i 
-$k/^i»  otherwise. 


4.  Tut  box  Texmlnatlon 

If  i - m,  terminate;  otherwise,  i *■  i + 1 and  go  to  1. 


At  the  termination 


•1  ml  1 m 

of  ALG  1,  B ■ E • • • E , and  each  factor  E , . . . , E , 


take  the  form  of  Z in  (1). 

In  the  following  two  propositions  we  show  that  if  in  step  2, 

- 0,  then  the  proposed  interchange  is  always  possible.  Consider  the 
following : 


Proposition  4. 

For  i < j,  EJ  •••  E1B(i)  - e1. 

Proof.  By  the  construction  of  E1  and  Proposition  3,  E •••£  B(i)  • 
e1.  By  Proposition  2,  Ej  •••  E1+1ei  - e1.  So  Ej  •••  eHu)  - e1. 
Using  Proposition  4 we  may  now  show  the  following: 


Proposition  5. 


For  2 £ i < m,  let  3 - E1"1  •••  E^d).  If  3±  - 0,  there  is 


_i-l 


E^ ( j ) ] ± 0. 


j > i such  that  [E 

Proof.  Suppose  [E1-1  •••  eVj)^  - 0 for  all  j > i.  By  the 
construction  of  E1,  ...  E1”1,  in  ALG  1,  and  Proposition  1,  each 
factor  is  nonsingular.  Since  B is  nonsingular,  E _1  ...  EXB  is 
nonsingular.  By  Proposition  4,  E ...  E B(j)  ■ e^  for  1 £ j £ ^ 

Hence,  the  ith  row  of  Ei_1  ...  ELB  is  all  zero,  a contradiction. 


III.  BUMP  AND  SPIKE  STRUCTURE 


In  order  to  minimize  the  core  storage  required  to  represent  the, 

ETA  file,  i.e.  E1,  ....  Em,  the  rows  and  columns  of  B are  interchanged 

in  an  attempt  to  place  B in  lower  triangular  form.  If  this  can  be  accom- 

2,  in 

plished,  then  the  m nonidentity  columns  of  E , ...»  E , have  the  same 
sparsity  structure  as  B.  Consider  the  following  proposition: 

Proposition  6. 

If  the  first  j-1  components  of  B(j)  are  zero  for  j > 2,  then 
EJ_1  ...  EVj)  - B(j). 

Proof.  This  follows  directly  from  successive  application  of  Proposition  2. 
Therefore,  if  B is  lower  triangular,  the  factored  representation  of  B 
may  be  stored  in  approximately  the  same  amount  of  core  storage  as  B 
Itself.  In  practice  it  is  unnecessary  to  calculate  the  elements  1/3^ 
and  -3^$^  in  step  3 of  ALG  1.  It  sufficies  to  store  k and  the  elements 
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of  6^.  It  may  prove  advantageous  to  store  * in  addition. 

If  Proposition  6 applies  for  B(k),  then  @ ■ B(k)  and  the  only  additional 
storage  required  is  for  the  index  k (and  possibly  1/B^) . Clearly, 
this  results  in  substantial  core  storage  savings  compared  to  storing 
B-1  explicitly. 

If  B cannot  be  placed  in  lower  triangular  form,  then  it  is  placed 
in  the  form: 


3 

where  B and  B are  lower  triangular  matrices  with  nonzeroes  on  their 

2 

diagonals.  We  assume  that  if  B is  nonvacuous,  every  row  and  column 

2 

has  at  least  two  nonzero  entries,  so  that  no  rearrangement  of  B can 

13  2 * • 

expand  the  size  of  B or  B . B is  called  the  bump  4e.ctA.on,  the  meAxt 

section  or  the  hea/it  section.  We  further  require  the  heart  section  to 
assume  the  following  form: 


where  G ' s are  either  vacuous  or  lower  triangular  with  nonzeroes  on 
the  diagonal.  The  only  partitions  in  B having  columns  with  nonzeroes 
above  the  diagonal  are  the  F^'s  which  are  called  external.  bumpA . The 
columns  extending  above  the  diagonal  are  called  ApikeA  or  A pike  coiumnA. 
An  external  bump  is  characterized  as  follows: 

(1)  the  last  column  of  an  external  bump  will 
be  a spike  with  a nonzero  lying  in  the 
topmost  row  of  the  external  bump,  and 
(il)  the  nonspike  columns  have  nonzero  diagonal 
elements. 

The  algorithms  of  Hellerman  and  Rarick  [3,  4]  produce  such  a structure 
for  any  nonsingular  matrix  B,  and  we  shall  call  a matrix  having  this 
structure  an  HR  matrix.  It  should  be  noted  that  if  one  applies  ALG  1 
to  an  HR  matrix,  then  the  only  columns  which  may  require  an  interchange 
are  spike  columns,  (tie  now  prove  that  the.  only  coiuirm  which  need  be 
conAidered  fan.  thiA  interchange  one  other  4 pike*  in  the  tame  external 


bump. 


Consider  the  following  result: 


Proposition  7 . 


Let  B(i)  with  i 2 correspond  to  the  first  column  of  some  external 
bump,  F^,  and  let  B(j)  be  a spike  in  F^.  Then  E*  ^ ...  E^B(j)  ■ B(j). 
Proof.  Note  that  the  first  i - 1 components  of  B(j)  are  zero. 
Therefore,  by  successive  application  of  Proposition  2,  the  result 
is  proved. 

Note  that  Proposition  6 allows  one  to  eliminate  all  of  the  calculation 
required  in  Step  1 of  ALG  1 for  non-spike  columns  and  Proposition  7 
allows  one  to  eliminate  some  of  this  calculation  for  spikes.  We 
now  address  the  issue  of  spike  swapping.  Consider  the  following 


propositions : 
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Proposition  8. 


Any  spike  B(j)  which  is  not  pivotable  cannot  be  interchanged 
with  a spike  B(k),  k > j,  from  another  external  bump,  to  yield  a 
pivotable  column. 

Proof.  Since  B(k)  is  from  an  external  bump  lying  to  the  right  of 
the  external  bump  containing  B(j) , Bj (k)  - 0.  By  repeated  application 
of  Proposition  2,  Ej_1  ...  E1B(k)  - B(k).  Thus  B(j)  cannot  be  inter- 
changed with  fl(k ; to  yield  a pivotable  column. 

Proposition  9. 

Any  spike  B(j)  which  is  not  pivotable  cannot  be  interchanged 
with  a non-spike  column  B(k),  k > j,  to  yield  a pivotable  column. 
Proof.  Let  B(k),  with  k > j correspond  to  any  non-spike  column. 

From  Proposition  6,  E^"1  ...  E^k)  - B(k).  Since  the  jth  com- 
ponent of  B(k)  is  zero,  B(j)  cannot  be  interchanged  with  B(k),  to 
yield  a pivotable  column. 

Proposition  10. 

Any  spike  column  B(j),  which  is  not  pivotable  can  be  interchanged 
with  a spike,  B(k),  with  k > j within  the  same  external  bump,  to 
yield  a pivotable  column. 

Proof.  If  B(j)  is  not  pivotable,  then  by  Proposition  5 there  exists 
a column  B(k)  with  k > j which  is  pivotable.  By  Proposition  8, 

B(k)  cannot  be  a spike  from  a different  external  bump.  By  Propo- 
sition  9,  B(k)  cannot  be  a non-spike.  Hence  B(k)  must  be  a spike 
from  the  same  external  bump. 

We  now  revise  ALG  1 incorporating  the  results  of  Propositions 
6,  7,  anc.  10. 


i 
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ALG  2:  PROVUCT  FORM  FACTORIZATION 
FOR  A HR  MATRIX 

0.  Initialization 

(same  as  ALG  1) 

1.  Update  Spike 

Let  B(k)  correspond  to  the  first  column  of  the  external  bump 
containing  B(i).  Set  0 ■♦*  E*  ^ • • • E^B(i). 

2.  Swap  Spike A 1)5  P-ivo-t  E£emen£  Equate  Zeno 

If  0^  + 0,  go  to  3;  otherwise,  there  is  some  spike  B(j)  in  the 
same  external  bump  having  j > i such  that  the  i^  component  of 
E1"1  •••  E^tj)  is  nonzero.  Interchange  B(j)  and  B(i)  and  return 
to  1. 

3.  Obtain  Mew  Column.  Eta 
(same  as  ALG  1) 

4.  Tut  ion.  Tenmination 

If  i ■ m,  terminate;  otherwise,  i •*-  i + 1. 

5.  Tut  ion.  Spike 

If  B(i)  Is  a spike,  go  to  1;  otherwise,  set  0 «-  B(i)  and  go  to  3. 

The  results  of  this  section  also  hold  if  one  uses  the  elimination 

form  of  the  inverse,  which  corresponds  to  the  Gauss  reduction.  The 

difference  between  the  product  and  the  elimination  form  factorizations 

is  that  for  the  latter  one  develops  2m  - 1 triangular  factors  such  that 

B'1  ■ • • • E . A factor  of  the  form  shown  in  (1)  is  called  tniangulan. 

if  either  z ■ z.  , « 0 or  z,.,  - ...  - z - 0.  The  details  of 
1 j-i  j+1  «n 

the  elimination  form  algorithm  are  given  in  [ 2] . 


| 
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